PCT 



WORLD INTELLECTUAL PROPERTY ORGANIZATION 
tat Burei 




International Bureau 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 6 : 
G01B 11/03, 11/24, G01C 11/28 



Al 



(11) International Publication Number: WO 98/04881 

<43) International Publication Date: S February 1998 (05.02.98) 



(21) International Application Number: PCT/NO97/00189 

(22) International Filing Dale: 21 July 1997 (21.07.97) 



(30) Priority Data: 
963047 



22 July 1996 (22.07.96) 



NO 



(71) Applicant (for all designated States except US): METRONOR 

AS INO/NO); Fekjan 13. Nesbru Senter. N-1360 Nestmi 
(NO)., 

(72) Inventors; and 

(75) Inventors/Applicants (for US only): PETTERSEN. Alf 
(NO/NO]; Emii Stangs vei 27B. N-1346 Gjettum (NO). 
R0TVOLD. Oyvind (NO/NO); Lidarende 1, N-1364 
Hvalstad (NO). 

(74) Agent: LANGFELDT. Jens, F„ C; Bryns Patentkontor A/S. 
P.O. Box 765. Sentrom. N-0106 Oslo (NO). 



(81) Designated Stales: AL. AM, AT. AU. AZ, BA. BB. BG. BR. 
BY. CA. CH, CN, CU. CZ. DE, DK, EE. ES. FI. GB. GE, 
GH. HU. IL. IS. JP. KE. KG, KP. KR, KZ, LC, LK, LR, 
LS. LT. LU. LV. MD. MG. MK. MN. MW. MX, NO. NZ. 
PL. PT. RO, RU. SD. SE. SG. SI, SK, SL. TJ, TM. TR. 
TT, UA, UG, US, UZ, VN. YU. ZW. ARIPO patent (GH, 
KE, LS. MW. SD. SZ, UG, ZW). Eurasian patent (AM. AZ. 
BY. KG. KZ. MD. RU. TJ. TM). European patent (AT. BE, 
CH. DE. DK. ES. FI. FR. GB. CR. IE, IT. LU. MC. NL, 
PT. SE). OAPI patent (BF. BJ. CF. CG. CI. CM, GA. GN, 
ML, MR. NE. SN. TD. TG). 



Published 

With international search report. 

In English translation (filed in Norwegian). 



(54) Title: SYSTEM AND METHOD FOR DETERMINING SPATIAL COORDINATES 

(57) Abstract 

The invention relates to a system for de- 
termining spatial coordinates of points, wherein 
the system involves a fixed array of a network 
(2) of control points (3). and a manually or me- 
chanically manipulatable measuring probe (I; 
I") containing at least one camera (9\ 9*', 9 ,,, ) 1 
and wherein the measuring probe (I; 1') is de- 
signed for physical contact with a measuring 
point on an object (4) which is to be measured 
via a fixedly mounted contact unit (14) pro- 
.ecting from the measuring probe, which has a 
known, defined position in the coordinate sys- 
tem of the measuring probe: and a system unh 
(5) which computes the position of the contact 
unit (14) relative to a coordinate system defined 
by said control points (3) on the basts of the 
measuring probe's (I; I') image of the control 
points (3), and the known position of the contact 
unit (14) in the coordinate system of the mea- 
suring probe (1; I '). There arc also methods for 
calibrating the measuring probe, determining the 
position of control points (3) in a network (2) 
and calibrating the contact unit (14) on the mea- 
suring probe (1; V). 
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SYSTEM AND METHOD FOR DETERMINING SPATIAL COORDINATES 

The present invention relates to a system for determining spatial coordinates of points, 
and methods related to the calibration of a probe and its contact unit, and to the 
5 determination of the position of points in a network. 

Norwegian Patent No. 174 025 describes a system for determining spatial coordinates 
of points on a face or an object, where the measuring probe is made in the form of a 
light peri having light sources at known coordinates relative to a probe-fixed coordinate 
1 0 system, and where the spatial coordinates are determined by registering the image of the 
light sources on a sensor in a single stationary camera. The position of the light pen is 
thus determined in a coordinate system given by the position of the camera. The system 
results in limited accuracy, given by the dimensions of the light pen and the visual field 
of the camera. 

15 

Norwegian Patents Nos. 164 946 and 165 046 describe systems and methods based on 
two or more stationary or movable cameras. Great accuracy is attained by using these in 
that the measuring point is observed from two or more camera positions. 

20 The object of the present invention is to redress the limitations associated with the 

known solutions. The system according to the present invention is thus characterised by 
the features which are set forth in the patent claims below. The methods mentioned 
above are also characterised by the features set forth in the patent claims below. These 
and additional distinctive features of the invention will also be set forth in the following 

25 description with reference to the appended drawings. 

New, miniaturised cameras make possible a hand-held probe containing a camera, and 
where its position and orientation are determined on the basis of registering an image of 
a fixedly mounted network of reference points. The difference from the previous 

30 system as described in Norwegian Patent No. 1 74 025 is thus that a network of reference 
points is stationary relative to the object which is to be measured, whilst the probe, 
which according to the present invention contains one or more cameras, is moved across 
the object which is to be measured. Considerably higher accuracy is attained with this 
solution than with the previously patented system based on one camera and light pen. 

35 This is due to the fact that images of several points can be made at each registration, and 
their geometrical distribution can be adapted such that they are distributed across the 
entire visual field of the camera and at different distances from the camera-based probe. 
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15 



The accuracy attainable with this system solution is comparable to what was previously 
attainable with a minimum of two cameras which view the object which is to be 
measured from two different directions of observation. This invention thus provides a 
5 system which is far simpler, is low in weight and complexity, and low in production 
costs. 

The probe can be made wireless in that the image is transmitted from the camera to a 
central unit by radio signals or infrared communication. 

10 

Figures la and lb show the measuring system and the use thereof. 

Figures 2a, 2b and 2c show a camera-based probe respectively in side view, top 
view and from below. 

Figure 3 shows a simplified, ideal camera model. 

Figure 4 shows determination of camera position in a network 

having known coordinates. 

Figure 5 shows how the position of the contact unit can be 

determined. 

Figure 6 shows a probe with several camera units. 

Figure 7 illustrates the use of the invention for three-dimensional 

measuring in connection with a three-dimensional object, 
e.g., a car model. 

30 Figure 8 shows the principle of wireless transmission of signals 

between probe and system unit. 

Figure 9 shows the probe mounted on a robot. 

35 The measuring system and the use thereof are outlined in Figures 1 a and 1 b. The 

system consists of a measuring probe 1 based on one or more cameras in one unit, and a 
network 2 of control points 3, a system unit containing computer unit 5 and an operator 



20 



25 
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console 6, 7. Usually, the operator console will consist of a keyboard 6 and a monitor 7. 
The measuring probe 1 is normally connected to the system unit 5 via a cable 8. The 
principle of measurement is that the camera-based probe 1 touches the object 4 at the 
point which is to be coordinate-determined, that the camera in this position makes 
5 images the whole of or parts of the network 2 of control points 3, and that the spatial 
position of the probe is computed on the basis of the registered image. ( 

As shown in Figs. 2a, 2b and 2c, the probe consists of a camera 9 with handle 12 
mounted thereon, exposure button 13 and contact unit 14. Normally, the camera is a 

10 video camera 10, e.g., based on CCD (Charge Coupled Device) or CID (Charge Injected 
Device) sensor technology, with lens 1 1 . The exposure button 1 3 is manipulated to 
actuate an image making of the network 2 of control points 3. The contact unit 14 is a 
replaceable unit, e.g., in the form of a tip or a sphere. The position of the probe I is 
related to the centre of the contact unit 14. This presupposes that the position of the 

15 centre is known relative to the coordinate system of the probe 1 . Method for this are 
described below. If the contact unit 14 is a sphere, the analysis of the measurement 
results must include compensation for the radius of the sphere. This could be avoided 
by instead using a tip-shaped contact unit. 

"20 According to the camera model which is outlined in Figure 3, the optics 1 1 of the 
camera 9 may be represented as a tiny hole, a so-called "pinhole" 1 7. Images of all 
points 3 are recorded on the sensor 10 as an ideal rectilinear projection through this hole 
1 7 (the centre of projection) of an image point 18. In a practical camera, lens error such 
as distortion will involve this straight line from the object point being deflected through 

25 the lens. It is possible to compensate for the error which occurs as a result of this 
deviation from the simplest camera model by calibrating the camera. In this context 
calibrating means determining the relation between the true image of a point on the 
sensor, and the spatial direction to the point relative to a camera-fixed coordinate 
system. 

30 

A network 2 of control points 3 is mounted around the object which is to be measured. 
In Figures 1 and 2 control points are only given in one plane on one side of the object 
which is to be measured. It would be advantageous to have control points on several 
sides of the object, optionally above and/or below, and at different distances from the 
35 object. This would give improved accuracy and ensure unambiguous mathematical 

solutions. The control points are so formed that they are easily identifiable in relation to 
the background. This may be as points or patterns (circles, crosses) painted in white on 
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a black background, rctroreflcctivc measuring points, so-called targets, or active light 
sources (e.g., light emitting diodes). Other easily identifiable patterns may also be used. 
To register retroreflective targets the camera or probe should have a powerful light 
source mounted thereon, e.g., a flash unit 15, which illuminates the points 3 when 
5 creating images. When using active light sources instead of retroreflective targets, these 
may be on continuously, or may be switched on synchronously with the exposure 
control of the camera. 

The network 2 may be mounted on a movable object, e.g., a frame which is brought to 
1 0 the object that is to be measured. 

The position of the individual control points 3 in the network 2 must be known with the 
greatest accuracy possible in order to be able to compute the position of the probe 1 on 
the basis of a single image. Methods for determining the position of the control points 3 
1 5 in relation to one another are described below. 

The determination of camera position relative to the network 2 of points having known 
coordinates will now be described in more detail. To understand this description it is 
essential to distinguish between the following coordinate systems: 

20 

• The network's spatial coordinate system. The coordinates of the network points 3 
are known relative to this coordinate system. 

• The internal coordinate system of the camera (probe). All registrations by the 
25 camera's sensor are related to this coordinate system. 

• The coordinate system of the object which is to be measured. When measuring, e.g., 
points on a car body, it is desirable for these to be related to a car-fixed coordinate 
system. A coordinate system of this kind is defined by specific points on the object, 

30 where these points are defined as having certain coordinate values. 

The image of the network 2 of control points 3 gives a number of image points 1 8 on 
the sensor 10 of the camera as shown in Fig. 4. On the basis of these registered image 
points in the coordinate system of the camera (probe) and knowledge of the spatial 
35 coordinates of the control points 3 in a stationary, space-fixed coordinate system, it is 
possible to compute the position and orientation of the probe 1 relative to the stationary 
spaced-fixed coordinate system 2. 



BNSDOCID: <WO 9804881 A 1 I > 



WO 98/0488 1 PCT/N 097/00 189 



There are different methods for carrying out this computation. The simplest would be to 
make a direct coordinate transformation between the coordinates of the image points in 
the camera-fixed coordinate system and the associated known coordinates of the points 
5 in the network's coordinate system. An accurate and approved method consists of 
making a bundle adjustment as known from photogrammetric technique. 

An efficient computation presupposes that it is known or rapidly discovered which 
image points correspond to which points in the network. This may be based on the 
10 points in the network being arranged in easily identifiable patterns, and that standard 
algorithms for pattern recognition are used for identification. 

There are no limitations on the orientation of the probe 1 , but the accuracy of 
determination of its position and orientation will depend upon how many of the control 
1 5 points 3 are made images of, and the spatial distribution of these. The image making of 
network points must take place so quickly that vibrations or the operator's movements 
do not have any effect upon the accuracy of measurement. The exposure time will 
typically be about 1/1000 of a second. 

20 A method of calibrating a video camera to determine spatial direction is described in 
Norwegian Patent No. 165 046. Within the field of photogrammetry other methods are 
known that are based on a network of points in known or unknown, positions being 
photographed from a camera in different positions and orientations relative to the 
network. The calibration of the camera consists of a number of correction parameters 

25 being determined, so that the image of one point may be considered as an ideal 

projection through the projection centre of the camera. As an alternative to correction 
parameters, a calibration table could be established corresponding to the relation 
between the real and ideal image of a point. 

30 Photogrammetric calibration methods are described in, e.g., H.M. Kamara (Ed.): Non- 
topographic photogrammetry. Second Edition, 1987. 

Calibration of the contact unit relative to a probe-fixed coordinate system will now be 
described in more detail. When measuring a point, the position of the probe 1 is 
35 required to be related to the centre of the contact unit 14. This requires knowledge of 
the position of the contact unit 14 relative to the coordinate system of the probe 1 
(camera). A simple method of determining the position of the contact unit 14 is 
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outlined in Fig. 5. This consists of holding the contact unit 14 at a fixed point, whilst 
the probe itself is moved around this fixed point. A number of registrations (e.g., as in 
the illustrated example) of the position and orientation of the probe 1 (camera) relative 
to the network 2 are made for different orientations of the probe. By combining 
5 registered position and orientation of the probe 1 for each registration with the 

information that the point of contact 14 was held still, the position of the centre of the 
contact unit 14 can be determined relative to both the network's coordinate system 2, 
and to the internal coordinate system 1 of the probe. 

1 0 This method is so simple and so fast to carry out that it makes possible the use of 

replaceable contact units in order to allow varied lengths and orientations of the contact 
unit relative to the rest of the probe. 

The coordinates of the individual points 3 in the network 2 can be determined by 
1 5 making images of the control points 3 from different directions by means of the probe 
unit 1. This requires the observation of each individual control point 3 from a minimum 
of two different directions. This procedure can be carried out as a separate process by 
using a precalibrated probe unit, or can be combined with the calibration of the probe. 

20 To be able to give the network's dimensions a correct scale, the distance between a 
minimum of two of the points in the network must be known, so that it can used as a 
scale factor in the bundle adjustment. If there are no known distances between fixed 
points in the network, it is possible by measuring the network to provide a distance 
standard in the network. The distance standard may be in the form of a rod 2 1 having 

25 control points of the same type as in the network. The distance between these control 
points must be known. 

The accuracy in the determination of the position and orientation of the probe will be 
influenced by a number of geometrical factors: 

30 

• dimension of the network, and distance between probe and network 

• density and form of control points in the network 

• the visual field of the camera (aperture angle) 

35 Improved accuracy may be attained by mounting several camera units with associated 
lenses 11 \ 1 1", 1 1 w together in a single probe as shown in Figures 6a and 6b. These 
may have wholly or partly overlapping visual fields as indicated in Fig. 6a, or different 
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visual fields as shown in Fig. 6b. The whole probe will have the same basic mode of 
operation as described above for a probe containing a single camera. The observations 
from the individual cameras will all be related to a common probe-fixed coordinate 
system. The image making of the network must be done synchronously by the 
5 individual cameras. 

: The calibration method described for a probe based on one camera can be extended to 
handle several cameras in one unit. These will have independent calibration parameters. 
In addition, position and orientation of the individual cameras can be determined 
1 0 relative to a common probe-fixed coordinate system. 

Figure 7 shows how the probe 1 1 can be used to measure points on the surface of a car, 
and wherein the network 2 is divided into several sub-networks 2\ 2" and 2"* of points 
3. 

15 

To avoid problems with cabling to the probe, the probe can be made wireless by 
mounting therein a module 16, 17 to transmit the image to a receiver module 18, 19 on 
the central unit via radio signals or infrared communication. It is not necessary to feed 
control signals back to the probe unit. Figure 8 shows the probe 1 connected to a 
20 transmitter 16 having antenna 1 7, and where the system unit 5 is equipped with a 
receiver 1 8 having antenna 19. Although the configuration is intended primarily for 
radio signals, it will be understood that the arrangement with some modifications is just 
as suitable for infrared communication. 

25 The measuring method can be automated by mounting the probe on a robot 20 or 

another mechanical device to guide the probe to a number of selected measuring points. 
The probe 1 may contain a sensor unit 21 for registering that the contact unit 14 is in 
physical contact with the object. When contact is registered, the position and 
orientation of the probe are automatically registered by making an image of the network 

30 2. 
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Patent claims 
1. 

A system for determining spatial coordinates of points, wherein the system involves a 
5 fixed array of a network (2) of control points (3), characterised by 

a manually or mechanically manipulatable measuring probe (1; V) containing at 
least one camera (9' 9", 9"*) and wherein the measuring probe (I; 1') is designed for 
physical contact with a measuring point on an object (4) which is to be measured via a 
fixedly mounted contact unit (14) projecting from the measuring probe, which has a 
10 known, defined position in the coordinate system of the measuring probe; and 

a system unit (5) which computes the position of contact unit (14) relative to a 
coordinate system defined by said control points (3) on the basis of the measuring 
probe's (1 ; V) image making of the control points (3), and the known position of the 
contact unit (14) in the coordinate system of the measuring probe (1 ; V). 

15 

2. 

A system as disclosed in Claim 1 , characterised in that 

the measuring probe (1 ; 1') contains a transmitter unit (1 6, 1 7) for wireless 
transmission of image and control signals to a receiver unit (18, 19) on the system unit. 

20 

3. 

A system as disclosed in Claim 1 , characterised in that 

the measuring probe contains at least two cameras (9\ 9", 9'"). 

25 4. 

A system as disclosed in Claim 3, characterised in that 

the cameras (9\ 9", 9 m ) have at least partly overlapping visual fields. 

5. 

30 A system as disclosed in Claim 3, characterised in that 

the cameras (9\ 9", 9"*) have separate visual fields. 

6. 

A system as disclosed in Claim 1 , characterised in that 
35 - the measuring probe (1 ; V) has a contact sensor (20) which is functionally 

connected to the contact unit (14) for registering mechanical contact between the contact 
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unit (14) and the object (4) which is to be measured at the measuring point, the contact 
sensor (20) on said registration causing automatic position measurement. 

7. 

5 A system as disclosed in Claim 1 , characterised in that 

the measuring probe (1 ) is equipped with a powerful light source, e.g., a flash 
unit (15). 

8. 

1 0 A system as disclosed in Claim 1 , characterised in that 

said control points (3) consist of one or more of the following types: active light 
sources, passive light sources, light reflecting targets, easily identifiable objects or 
patterns. 

15 9. 

A system as disclosed in Claim 1 , characterised in that 

said network (2) of control points (3) is mounted on a movable object. 

10. 

20 A method for calibrating a measuring probe in a system as disclosed in Claim I , 
characterised by 

moving the camera (9) or cameras (9\ 9", 9'") of the measuring probe (1 ; P) to 
a number of different observation directions or orientations; 

successively recording in the camera or cameras images of the fixed array of the 
25 control points (3) in the network (2) from said different observation directions or 
orientations; and ■ 

correlating the successive images with one another to establish a number of 
correction parameters or a calibration table corresponding to the relation between the 
real and ideal image of a point which is due to the optical properties of the camera or 
30 cameras. 

11. 

A method as disclosed in Claim 10 and wherein the measuring probe has at least two 
cameras (9\ 9", 9'"), characterised by 
35 - causing the optical properties of each camera to be determined by bundle 
adjustment. 
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12. 

A method for determining the position of control points (3) in a network (2) which is 
part of a system as disclosed in Claim 1, wherein the distance between at least two of 
the control points is known, characterised by 
5 - moving the camera (9) or cameras (9\ 9", 9"') of a calibrated measuring probe 
(1 ; V) to at least two different directions of observation; 

recording in the camera or cameras an image of the fixed array of the control 
points (3) in the network from said different directions of observation; and 

establishing a table of coordinate values for the control points (3) in a coordinate 
1 0 system for the network (2) defined by a selection of the control points (3). 

13. 

A method as disclosed in Claim 12, characterised by 

causing the spatial coordinates of the control points to be determined by bundle 
1 5 adjustment. 

14. 

A method for calibrating a measuring probe (1 ; V) in a system as disclosed in Claim 1 , 
where the position of control points (3) in a network (2) which form part of the system 
20 are determined concurrently, the distance between at least two of the control points 
being known, characterised by: 

moving the camera (9) or cameras (9\ 9", 9"') of the calibrated measuring probe 
(1 ;T) to at least two different directions of observation; 

recording in the camera or cameras an image of the fixed array of the control 
25 points (3) in the network (2) from said different directions of observation; and 

establishing a table of coordinate values for the control points (3) in a coordinate 
system for the network (2) defined by a selection of the control points (3) and in the 
same computation correlating the images for establishing a number of correction 
parameters or a calibration table corresponding to the relation between the real and ideal 
30 image of a point. 

15. 

A method for calibrating the contact unit (14) on a measuring probe (I ; ! ') in a system 
as disclosed in Claim 1, characterised by: 
35 - moving the camera (9) or cameras (9\ 9", 9"') of the measuring probe (I; V) to 
a number of different observation directions or orientations, whilst its contact unit (14) 
remains in contact with a fixed reference point or measuring point (4*) on an object (4); 
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registering the position and orientation of the measuring probe (1 ; V) relative to 
a network (2) of control points (3) in that in the camera or cameras there is recorded an 
image of the fixed array of the control points (3) in the network for each of said 
directions or orientations; and 
5 - combining registered position and orientation of the measuring probe (1 ; P) for 
each registration with information that the point of contact (4 1 ) for the contact unit (14) 
is the same, whereby the position of the centre of contact unit (14) is determined in 
relation to the network's coordinate system and the interna] coordinate system of the 
. measuring probe ( 1 ; 1'). 

10 
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